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1 • 0  INTRODUCTION 

^This  report  describes  the  technical  work  accomplished  in  the 
second  phase  of  a  program  to  develop  an  automated,  three-dimensional, 
quantitative  surface  topography  measurement  system  using  a  stereo-- 
pair  of  scanning  electron  microscope  (SEM)  micrographs.  The  system 
embodies  the  disciplines  of  stereo-microscopy  and  stereo-photogram- 
metry.  The  first  phase  of  the  program  (ONR  Contract  N00014-79-C-0792) 
explored  the  technical  feasibility  of  making  quantitative  three- 
dimensional  measurements  of  surface  topography  using  a  stereo-pair  of 
SEM  micrographs.  The  objectives  of  the  second  phase  of  the  program 
(ONR  Contract  N00014-80-C-0937)  were  to  develop  the  hardware  and 
software  necessary  to  implement  the  surface  topography  measurement 
system. 

1.10  Summary  of  Work  Done  in  Phase  I 

Phase  I  consisted  of  a  comprehensive  literature  survey  to 
establish  the  state-of-the-art  in  the  disciplines  of  stereo-micro¬ 
scopy  and  stereo-photogrammetr y  (Task  1)  and  a  set  of  experiments  to 
establish  the  feasibility  of  the  proposed  surface  measurement  system 
(Task  2) . 

The  results  of  interest  from  Phase  I  are  summarized  below: 

1)  SEM  stereology  lias  been  used  successfully  in  the  biological 
sciences.  Major  sources  of  error  have  been  identified  and 
procedures  have  been  developed  to  circumvent  these  problems. 
Some  commercial  systems  are  available  for  three-dimen¬ 
sional,  quantitative,  surface  topographical  measurements 
using  a  stereo-pair  of  SEM  micrographs.  These  systems 
require  the  operator  to  identify  the  sets  of  matched  points 
used  to  calculate  surface  heights.  The  results  are,  there¬ 
fore,  operator  dependent. 
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2)  Automated  digital  cartographic  systems  using  a  stereo-pair 
of  digitized  images  have  been  developed.  Development  of 
digital  stereo-photogrammetry  was  motivated  by  the  need  to 
extract  topographical  information  from  terrain  data  obtained 
from  a  wide  range  of  sensors.  These  systems  are  either 
general  purpose  in  nature  requiring  large  mainframe  com¬ 
puters  or  have  been  implemented  using  special  purpose 
hardware . 

3)  The  proof-of-concept  tests  were  successful.  Surface  heights 
measurements  using  a  stereo-pair  of  SEM  micrographs  were 
within  10%  of  the  measurements  using  stylus  instruments. 

Phase  I  resulted  in  the  definition  of  a  surface  measurements 
system  combining  the  developments  in  SEM  stereology  and  digital 
stereo-photogrammetry.  The  proposed  system  consisted  of  an  SEM  with 
a  computer  controlled  scan  generator  and  the  hardware  to  digitize  the 
SEM  video  signal.  To  digitize  a  stereo-pair  of  SEM  images,  the 
specimen  to  be  characterized  is  oriented  in  the  initial  position  and 
the  computer  is  used  to  scan  the  specimen  and  to  digitize  and  store 
the  first  image.  The  specimen  is  reoriented  (tilted,  translated  or 
combination)  in  the  second  position  and  the  second  image  is  digitized 
and  stored.  Scan  software  is  employed  to  coordinate  the  sampling  of 
the  SEM  video  signal  with  the  scan  generator  signal.  The  digitized 
stereo-pair  of  SEM  images  are  then  processed  by  the  image  matching 
software  to  generate  a  three-  dimensional  map  of  the  surface. 
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1.20  Objectives  of  Phase  II 

The  development  of  the  hardware  and  software  needed  to  implement 
the  proposed  surface  measurement  system  was  divided  into  four  tasks: 


Task  1  - 

Task  2  - 
Task  3  - 


Task  4  - 


Select,  procure,  and  prove  the  hardware  required  to 
digitize  SEM  images. 

Develop  the  software  necessary  to  digitize  SEM  images. 
Develop  the  image  correlation  software  nr  e3  to  extract 
quantitative  surface  height  information  a  stereo-- 

pair  of  digitized  SEM  images. 

Verify  the  complete  surface  measurement  ■<  am. 


1.30  Organization  of  the  Report 

This  report  describes,  by  task,  the  work  accomplished  in  the 
second  phase  of  the  program.  The  selection  and  shakedown  of  the 
system  hardware  is  described  in  Section  2.  The  development  of  the 
software  needed  to  digitize  SEM  images  is  detailed  in  Section  3. 
Section  4  contains  a  description  of  the  image  matching  software. 
Tests  to  verify  the  performance  of  the  integrated  surface  measurement 
system  are  described  in  Section  5.  Section  6  summarizes  the  work  done 
in  Phase  II. 


2 . 0  PROCUREMENT  AND  SHAKEDOWN  OF  SYSTEM  HARDWARE  (TASK  1) 

The  hardware  system  needed  to  digitize  SEM  images  was  defined. 
The  specific  components  of  the  system  were  selected,  procured,  and 
integrated  into  one  system.  The  activity  in  this  task  is  described 
below. 
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2.10  Selection  of  System  Hardware 

The  purpose  of  this  task  was  to  procure  and  prove  the  hardware 
needed  to  digitize  SEM  images.  Digitizing  SEM  images  required  the 
operation  of  SEM  scan  coils  under  computer  control  and  the  ability  to 
digitize  the  output  from  the  SEM  video  amplifier.  The  hardware,  and 
associated  software,  selected  for  the  purpose  included: 

1)  A  PDP-11/23  micro-computer  system  with  128K  bytes  of  memory, 
two  RL02  disc  drives  with  a  storage  capacity  of  10  megabytes 
per  disc,  a  VT-100  console  terminal,  and  a  LA-180  printer. 
The  system  is  operated  using  the  RT-11  (version  4)  operating 
system.  The  software  is  written  in  FORTRAN  IV  (version  2.5) 
and  in  PDP-11  assembly  language  using  the  MACRO-11  (version 
4)  assembler. 

2)  Tektronix  4662  interactive  digital  plotter  and  Tektronix 
4010  interactive  terminal  were  selected.  The  graphics  hard¬ 
ware  is  driven  by  the  Tektronix  PLOP-10  software  package. 

3)  Real-time  clock  system.  Data  translation  DT-2769  real-  time 
clock  was  selected.  The  clock  is  fully  compatible  with  the 
KWV-11  real-time  clock  made  by  Digital  Equipment  Corporation 
(DEC)  . 

4)  Analog-to-Digital  Converter.  Data  translation  DT-2782  12-- 
bit  A/D  converter  was  selected.  The  converter  comes  with  a 
self-contained  DMA  interface  for  high  speed  sampling  of 
analog  data  and  storage  anywhere  within  an  18-bit  address 
space  of  the  PDP-11/23  computer  system. 

5)  Digital-to-Analog  Converter.  Data  translation  DT-2766  4-- 
channel  12-bit  D,/A  converter  was  selected.  The  system  is 
software  compatible  with  the  AAVll-A  converter  manufactured 
by  DEC. 

6)  A  library  of  real-time  peripheral  support  software  (DTLIB/RT 
version  2.2)  was  selected  to  control  the  data  translation 
interface  board  listed  above.  The  library  is  compatible  with 
RT-11  (version  4)  operating  system  and  FORTRAN  IV  (version 
2.5)  supplied  by  DEC. 
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The  graphics  hardware  and  software,  the  data  translation  inter¬ 
face  boards,  and  the  DTLIB  software  library  were  integrated  with  the 
computer  system.  The  integrated  system  was  tested  to  verify  the 
operation  of  each  of  the  components.  in  addition,  an  interface  was 
built  to  allow  the  computer  system  to  access  and  control  the  SEM  scan 
hardware.  A  circuit  drawing  of  the  interface  is  included  in  Appendix 
A.  The  interface  provides  access  to  the  SEM  scan  coils  and  the  SEM 
beam  blanking  signal,  and  an  ability  to  augment  the  signal  sent 
from  the  SEM  video  amplifier  to  the  SEM  cathode  ray  tube  (CRT). 

2.20  System  for  Digitizing  SEM  Images 

A  schematic  of  the  system  for  digitizing  SEM  images  is  shown  in 
Figure  1.  SEM  images  are  formed  by  modulated  the  brightness  of  spots, 
referred  to  as  pixels,  on  the  SEM  CRT  in  proportion  to  the  output  of 
the  SEM  video  amplifier.  To  digitize  an  image,  the  voltage  signal 
from  the  SEM  video  amplifier  at  each  spot  on  the  specimen  is  converted 
into  an  integer  number  using  an  A/D  converter;  the  number  is  referred 
to  as  the  gray  level  of  the  pixel.  A  digitized  SEM  image  is  the  array 
of  gray  levels  generated  as  the  SEM  scans  a  specimen. 

The  SEM  is  operated  under  computer  control  to  obtain  a  digitized 
image.  The  SEM  beam  is  moved  to  the  desired  spot  on  the  specimen  using 
the  D/A  converter.  The  real-time  clock  is  then  used  to  trigger  the 
integrator  to  integrate  the  SEM  video  amplifier  output  for  a  specified 
time  interval.  After  completing  the  integration,  the  integrator 
signals  the  A/D  converter  to  convert  the  integrated  analog  signal  into 
an  integer  number  and  to  store  the  number  in  an  input  buffer.  The  SEM 
beam  is  then  moved  to  the  next  point  to  be  digitized.  The  software 
developed  to  digitize  SEM  images  is  described  in  Section  3. 
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3.0  SOFTWARE  FOR  DIGITIZING  SEM  T MAGES  (TASK  2) 

The  software  for  digitizing  SEM  images  consists  of  a  main  FORTRAN 
program  which  calls  a  set  of  assembly  language  subroutines.  Each 
assembly  language  subroutine  is  designed  for  a  specific  operation  and 
the  main  program  is  used  to  used  to  specify  the  order  in  which  these 
operations  are  performed. 


3.10  Description  of  the  Screen  Coordinate  System 
A  SEM  image  is  divided  into  a  two-dimensional  array  of  NXSCAN  x 
NYSCAN  pixels  as  shown  in  Figure  2.  A  set  of  pixels  with  the  same 
Y  coordinate  (parallel  to  the  X-axis)  are  referred  to  as  a  scan  line. 
Thus,  an  image  has  NYSCAN  scan  lines,  with  NXSCAN  pixels  in  each  scan 
lines.  The  area  to  be  digitized  is  defined  by  specifying  the  screen 
coordinates  (NXLEFT ,  NYTOP)  of  the  upper  left  corner  of  the  area,  the 
number  of  scan  lines  to  be  digitized  (NROW)  ,  and  the  number  of  pixels 
(NPIXEL )  to  be  digitized  in  each  scan  line.  The  distance  d  between 
each  pixel  along  the  X  axis  is  given  by 


(MAGNFY) (NXSCAN- 1)  COS (TILT) 

where  MAGNFY  is  the  magnification  for  the  SEM  image,  w  the  length  of 
traverse  for  a  scan  line,  and  TILT  the  angle  of  tilt  for  the  image. 
3.20  Digitizing  SEM  Images 

A  flow  chart  for  the  control  program  to  digitize  SEM  images  is 
shown  in  Figure  3.  The  sequence  of  operations  required  for  digit¬ 
izing  a  SEM  image  is  as  follows: 

1)  Input  data 

2)  Select  area  to  be  digitized 

3)  Set  the  integration  time  for  integrating  the  SEM  video 
amplifier  signal. 
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4)  Digitize  and  store  image 

5)  Obtain  screen  coordinates  of  points  of  interest  in  the  image. 
This  feature  is  used  to  obtain  sets  of  matched  points  for  the 
image  matching  software  described  in  Section  4. 

The  structure  of  the  program  permits  the  user  to  tailor  the 
sequence  of  operations  for  a  specific  application.  For  example,  if 
the  complete  image  is  to  be  digitized,  operations  for  area  selection 
can  be  left  out  of  the  sequence.  Each  of  the  five  operations  is 
described  below. 

3.21  Input  Variables 

Program  input  variables  are  listed  in  Table  1,  along  with  the 
range  of  acceptable  values  for  the  variables.  Variables  NXSCAN, 
NYSCAN,  NROW ,  NPIXEL,  MAGNFY ,  and  TILT  were  described  in  Section  3.10. 
The  values  of  NPIXEL  are  restricted  to  integer  multiples  of  256  to 
facilitate  data  storage  and  retrieval  from  random  access  disc  systems 
used  with  the  PDP-11  series  computers.  The  two  remaining  input 
•ariables  are  the  SEM  accelerator  voltage  (CVOLTS)  and  the  number  of 
samples  tNAV)  to  be  averaged  at  each  pixel.  The  gray  level  at  each 
pixel  is  an  average  of  NAV  samples  of  the  SEM  video  amplifier  signal 
at  that  point.  Sample  averaging  is  used  to  reduce  the  random  noise 
in  the  digitized  data. 

The  program  assigns  default  values  for  the  remaining  parameters. 
The  values  assigned  are  listed  in  Table  2.  The  upper  left  corner 
of  the  area  to  be  digitized  (Figure  1)  is  assigned  such  that  the  area 
is  centered  in  the  image.  These  values  are  changed  when  area 
selection  operations  are  performed  (see  Section  3.22).  The  starting 
value  for  the  integration  time  (IDWELL)  is  set  at  200  ps  by  the 
program.  The  actual  value  for  the  integration  time  is  set  in 
operations  described  in  Section  3.23.  The  length  of  traverse,  w, 
for  the  ETEC  Autoscan  SEM  is  fixed  at  88.9mm. 
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3.22  Selecting  the  Area  to  be  Digitized 

The  area  to  be  digitized  is  delimited  by  a  cursor  box  as  shown 
in  Figure  1.  The  size  and  the  location  of  the  cursor  box  can  be 
changed  to  select  the  portion  of  the  image  to  be  digitized.  The  size 
of  the  box  is  changed  by  increasing  or  decreasing  the  number  of  scan 
lines  (NROW)  to  be  digitized.  The  number  of  pixels  to  be  digitized 
in  each  line  (NPIXEL)  cannot  be  changed.  The  user  is  required  only 
to  position  the  cursor  box  as  desired;  the  program  keeps  track  of  the 
values  of  all  the  relevant  variables. 

3.23  Setting  Integration  Time 

The  signal  from  the  SEM  video  amplifier  is  integrated  to  reduce 
the  effect  of  noise  on  the  digitized  gray  levels.  The  magnitude  of 
the  integrated  analog  signal  is  proportional  to  the  time  of  inte¬ 
gration.  The  integration  time  should  be  long  enough  to  reduce  the 
effect  of  noise  on  the  SEM  video  signal  but  not  so  long  that  the 
magnitude  of  the  integrated  signal  is  outside  the  range  (  +  10  volts) 
of  the  A/D  converter.  The  selection  of  the  integration  time  has  been 
automated.  The  program  starts  with  an  integration  time  of  200 
and  scans  the  specimen  to  check  the  magnitude  of  the  integrated 
signal.  If  more  than  ten  pixels  are  outside  the  range  of  the  A/D 
converter,  the  time  is  reduced  by  1  |js  and  the  specimen  is  scanned 
again.  The  process  is  repeated  until  there  are  less  than  ten  points 
outside  the  range  of  the  A/D  converter. 

3.24  Digitizing  the  Area  Selected 

The  procedure  for  digitizing  a  SEM  image  is  shown  in  Figure  4. 
After  setting  the  dwell  time  no  additional  input  is  required  to 
digitize  the  area  selected  by  the  user.  The  contrast  and  the 
brightness  settings  on  the  SEM  should  be  set  so  that  most  of  the  gray 
levels  are  in  the  range  +200  to  +2047.  This  range  of  gray  levels 
yields  the  best  results  (see  Appendix  A). 
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3.25  Obtaining  Screen  Coordinates  of  Points  of  Interest 

This  part  of  the  program  is  used  to  obtain  the  screen  coordinates 
of  features  of  interest.  The  program  displays  a  cross-hair  cursor 
superposed  on  the  image  on  the  SEM  CRT.  The  cursor  is  moved  to  the 
point  of  interest  and  the  "C"  key  on  the  keyboard  is  pressed  to  obtain 
the  coordinates  of  the  point.  This  facility  is  used  to  obtain  sets 
of  matched  points  required  to  start  the  image  matching  algorithm 
described  in  Section  4. 

3.30  Instruct  ions  for  Using  the  Software 

Image  digitization  software  starts  by  typing  star'  up  instruc¬ 
tions  (figure  5)  on  the  console  screen.  The  i .  •  t  act  ions  for 

displaying  the  cursor  box  may  be  disregarded  if  tin  wnole  image  is  to 
be  digitized.  After  the  start  up  instructions  huv.-  been  complied 
t  fv.  program  requests  values  for  input  par  amt  :*-is  and  the  name 
:  file  in  which  the  digitized  values  are  to  be  si  u  od .  The  file 

name  can  have  up  to  six  ci.  n  acters  and  the  file  *  xt .  ...ton  up  to  three 
o!  .uactets.  If  a  file  with  the  name  arid  <  x « •  •  i .  .  t.  'pcciticd  by  the 
«  i  i  r;  already  stored  on  the  disc,  the  ptogium  piompts  t  tie  user  to 
=e  thet  name.  When  program  input  is  completed,  the  function 
sole  ci  ion  m-  ru  snows  in  Figure  6  is  displayed  on  the  screen.  The  user 
can  now  chose  to  pet  form  any  of  the  opet  at  ions  1  isted  in  the  menu .  The 
menus  for  each  of  the  operations  are  described  below. 

The  cursor  box  menu  shown  in  Figure  7  directs  the  selection  of 
the  area  to  be  digitized.  The  position  and  the  size  of  the  cursor 
box  are  changed  using  the  arrow  function  keys  and  the  program  function 
(PF)  keys.  The  program  does  not  allow  the  cursor  box  to  move  past 
the  edges  of  the  screen:  a  bell  rings  if  the  user  attempts  to  move 
the  cursor  box  off  the  screen  and  the  position  or  the  size  of  the 
cursor  box  is  not  changed.  The  integration  time  menu  shown  in  Figure 
8  explains  the  selection  of  the  integration  time  and  allows  the  user 
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to  photograph  the  area  to  be  digitized.  The  point  selection  menu 
shown  in  Figure  9  explains  how  the  arrow  function  keys  are  used  to 
obtain  the  screen  coordinates  of  the  points  of  interest.  The 
coordinates  are  typed  in  the  lower  left  corner  of  the  console  screen. 

There  is  no  menu  for  image  digitization  and  storage;  the  prompt  " . 

DATA  COLLECTION  IN  PROGRESS...."  appears  on  the  screen  while  the 
image  is  being  digitized.  The  program  returns  to  the  function 
selection  menu  after  each  operation  is  completed  (see  Figure  3) . 

3.40  System  Validation 

The  integrated  hardware  and  software  system  was  tested  to  verify 
the  repeatability  of  the  digitized  image  data  collected  by  the  system 
and  to  devise  a  procedure  that  would  result  in  a  stereo-pair  of 
digitized  SEM  images  suitable  for  the  image  matching  software  de¬ 
scribed  in  Section  4.  The  tests  and  the  results  are  detailed  in 
Appendix  A.  The  tests  led  to  several  modifications  of  the  software 
to  improve  the  signal  to  noise  ratio  for  the  digitized  data.  The 
tests  showed  that  the  SEM  brightness  and  contrast  controls  should  be 
adjusted  so  that  most  pixels  have  gray  levels  in  the  200  to  2047 
range.  If  the  specimen  does  not  permit  the  above  adjustment,  sample 
averaging  should  be  used  to  reduce  the  noise  in  the  data. 

4.0  IMAGE  MATCHING  SOFTWARE  (TASK  3) 

A  stereo-pair  of  digitized  SEM  images  can  be  used  to  generate 
three-dimensional  maps  of  rough  surfaces.  The  first  step  is  to  match 
the  independent  image  of  the  stereo-pair,  referred  to  as  image  A,  with 
the  dependent  image,  referred  to  as  image  B.  The  matching  process 
consists  of  selecting  a  point  on  image  A  and  then  locating  the 
conjugate  point  on  image  B.  The  matched  images  are  then  used  to 
extract  micro- topographical  data  for  real  surfaces. 

The  matching  algorithm  selected  for  use  in  this  study  is  a 
simplified  version  of  the  algorithm  developed  at  the  U.S.  Army 
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Engineer  Topographic  Laboratories  (UASETL)  and  used  by  Control  Data 
Corporation  (CDC)  to  develop  a  benchmark  system  to  extract  topo¬ 
graphical  information  from  terrain  data  obtained  from  a  wide  range  of 
sensors.  The  USAETL  algorithm  is  simplified  because  the  SEM  images 
can  be  regulated  to  obtain  images  with  known  contrast.  This  cannot  be 
done  with  automatic  terrain  data  collection.  Therefore,  the  matching 
requirements  for  SEM  images  are  not  as  severe  as  those  for  automated 
digital  cartographic  systems.  The  material  in  Sections  4.10  to  4.60 
has  been  extracted  from  the  report  by  Panton  et  al  (1978). 

4.10  Coordinate  Systems  for  a  Stereo-Pair  of  SEM  Images 

A  digitized  SEM  image  is  a  two-dimensional  array  of  pixel  gray 
scale  values  generated  as  a  SEM  specimen  is  scanned.  In  this  section, 
a  digitized  SEM  image  is  represented  by  NR  rows  of  pixels  spaced  at 
intervals  of  ay,  each  row  having  Np  pixels  spaced  at  intervals  of  aX. 
A  digitized  SEM  image  is,  therefore,  a  Np  x  NR  array  of  pixel  gray 
scale  values.  A  stereo-pair  of  digitized  SEM  images  is  obtained  by 
tilting  the  specimen  about  one  of  the  scan  directions. 

The  axes  systems  chosen  to  locate  points  in  images  A  and  B  are 
:  ..town  in  Figure  10.  A  point  (X,Y)  in  image  A  maps  as  point  (U,V) 
in  mu-jo  i<.  If  the  stereo-pair  of  SEM  images  are  obtained  by  tilting 
about  the  Y  axis,  there  is  no  patallax  along  the  Y  direction:  the 

parallax  due  to  surface  topography  is  along  the  X  direction  only. 
Therefore,  the  Y  coordinate  of  a  pixel  in  image  A  and  the  V  coordinate 
of  its  match  point  in  image  B  are  the  same.  The  matching  process 
for  a  pixel  (X,Y)  is  reduced  to  determining  the  U  coordinate  of  the 
matching  pixel  in  image  B.  However,  it  is  not  feasible  to  obtain 
statistical  correlation  between  individual  pixels  on  the  two  images. 
Therefore,  the  U  coordinate  of  the  matching  point  on  image  B  is 
calculated  by  correlating  an  array  of  pixels  surrounding  the  point 
( X , Y )  in  image  A  with  a  set  of  arrays  of  pixels  centered  at  the 
estimated  location  of  the  match  point  in  image  B. 
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Image  A  is  divided  into  a  g'id  of  arrays,  Nx  x  Ny  pixels;  the 
arrays  are  called  blocks.  A  series  of  blocks  with  the  same  X 
coordinate  is  called  a  column  while  a  series  of  blocks  with  the  same 
Y  coordinate  is  called  a  block  path.  The  division  of  image  A  into 
columns  and  block  paths  is  shown  in  Figure  10  and  a  typical  block  is 
shown  in  Figure  11.  The  matching  process  requires  determination  of 
the  match  points  for  the  centers  of  all  the  blocks  in  image  A. 

4.20  Correlation  Strategy 

To  find  the  match  point  for  a  block  (I,J)  centered  at  point 
(X , Y ) ,  the  estimated  location  of  the  match  point  (u)  is  calculated 
using  coordinate  information  from  previously  matched  blocks.  A  set 
of  Ncor  arrays  of  Nu  x  Ny  pixels  is  centered  at  the  estimated  location 
of  the  match  point  in  image  B;  the  arrays  are  called  correlation 
patches.  The  set  of  Ncor  correlation  patches  is  called  the 
correlation  search  area.  The  size  Nu  of  a  correlation  patch  is 
calculated  to  conform  with  the  surface  topography  around  block  (I,J) . 
The  calculation  of  Nu  is  called  patch  shaping. 

The  Nu  x  Ny  gray  scale  values  in  a  correlation  patch  are 
resampled  to  obtain  an  array  of  Nx  x  Ny  gray  scale  values;  the 
resampled  array  is  called  a  resampled  correlation  patch.  A  cor¬ 
relation  coefficient,  R,  is  calculated  for  the  block  (I,J)  and  each 
of  the  Ncor  resampled  correlation  patches  in  the  correlation  search 
area.  A  parabolic  correlation  function  is  calculated  using  the 
largest  correlation  coefficient  and  values  of  R  for  the  two  adjacent 
correlation  patches.  The  match  point  for  block  (I,J)  is  then 
determined  to  the  nearest  pixel  by  locating  the  maxima  of  the 
correlation  function. 

The  output  of  the  matching  process  is:  coordinates  (X,Y)  in 
image  A,  coordinates  (U,V)  in  image  B,  and  the  correlation  coefficient 
R  which  serves  as  a  measure  of  the  reliability  of  the  match.  The 
matching  process  is  outlined  in  Figures  12  and  13. 
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4 . 30  Predicting  the  Location  of  the  Match  Point  in  Image  B 
The  predicted  location  of  the  match  point,  Ujf  j  is  obtained  using 
a  rate  of  change  function,  AU/AX.  The  rate  of  change  function  is  the 
rate  at  which  the  U  coordinate  changes  for  a  unit  rate  of  change  in 
the  X  coordinate.  It  is  a  useful  criterion  for  predicting  the 
location  of  a  match  point  and  for  calculating  the  size  Nu  of  a  shaped 
correlation  patch  in  image  B.  The  rate  of  change  function  for  a  block 
is  a  measure  of  the  surface  slope  near  the  center  of  the  block.  The 
rate  of  change  function  is  unity  for  a  flat  surface,  less  than  one  for 
an  uphill  slope,  and  greater  than  one  for  a  downhill  slope.  The  effect 
of  surface  topography  on  the  rate  of  change  function  is  shown  in 
Figure  14. 

For  a  block  (I,J)  on  image  A,  the  predicted  location  of  the  match 

a 

point  in  image  B,  Ujfj,  is  estimated  using  the  relation 

Ui.J  -  “I-1.J  *  ("1‘TT  >J  ♦  ♦  W3<-|f>J  +  1UX  (2| 

where  is  the  U  coordinate  of  the  previously  matched  block 

( I — 1 , J )  in  column  1-1,  Wj,  W2  and  W3  are  weights  such  that 

W3  +  W2  +  W3  =  1.0  (3) 

and  (9U/3X)  terms  are  the  values  of  the  rate  of  change  function  given 
by  (see  Figure  15) 

=  ~  UI-1,J-1>  /  Ax 

=  (Ul-l,J  “  Ux-2,j)  /  AX  (4) 

(Tx')j+1  =  (Ui-GJ+l  ~  uI-2,J+l)  /  Ax 
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The  use  of  a  weighted  value  of  the  rate  of  change  function  in  equation 

A 

(2)  to  predict  the  location  of  the  match  point  Ujfj  couples  the 
topography  of  block  paths  J,  J-l  and  J+l.  The  extent  of  coupling  can 
be  changed  by  specifying  different  weights.  The  values  of  the  weights 
for  SEM  environment  will  have  to  be  determined  by  experience. 

4.40  Patch  Shaping  and  Resampling 

A  block  of  Nx  x  Ny  pixels  in  image  A  will  compress  or  expand  in 
image  B  depending  on  the  local  surface  slope.  The  change  in  size  due 
to  parallax  will  be  along  the  X  axis  only.  The  size  Nu  of  the  image 
B  patch  is  given  by 

Nu  =  [Wl  (J-Ulj  +  «2  +  "3  'tI’j  +  I1  "*  l5) 

where  the  variables  in  equation  (5)  are  the  same  as  in  equation  (2). 

A  shaped  correlation  patch  of  Nu  x  Ny  pixels  has  to  be  resampled 
to  obtain  an  array  of  Nx  x  Ny  numbers  to  enable  correlation  with  a 
block  in  image  A.  The  resampling  is  done  by  linear  interpolation 
between  the  two  adjacent  pixels  using  the  relation  (see  Figure  16) 


Gr  =  Gi  (1  -  D} 


G2  D> 


where  Gr  is  the  gray  scale  value  of  the  resampled  point,  Gj  and  G2  are 
the  gray  scale  values  of  the  two  adjacent  pixels,  and  Dx,  the 
normalized  distance  of  the  sampled  point  from  pixel  2. 
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4.50  Correlation  Coefficients 

A  block  (I,J)  on  image  A  is  correlated  with  all  of  the  Ncor 
resampled  correlation  patches  centered  at  the  predicted  location  of 
the  match  point  on  image  B.  The  choice  of  a  particular  correlation 
measure  is  determined  by  the  type  of  the  digitized  images  being 
matched.  For  stereo  matching  applications,  where  one  image  appears 
to  be  different  from  the  other,  the  linear  correlation  coefficient 
is  preferred  because  it  acts  as  a  filter:  uniform  additive  and 
multiplicative  gray  scale  changes  do  not  effect  the  correlation 
coefficient.  The  correlation  coefficient  R  is  given  by 


where  Aij  is  the  pixel  gray  scale  value  for  pixel  (i,j)  in  a  block 
in  image  A;  A  the  average  of  all  the  pixel  gray  scale  values  in  the 
block;  B i j  the  gray  scale  numbers  in  a  resampled  patch  in  the 
correlation  search  area  in  image  B;  and  5  the  average  of  all  the  gray 
scale  numbers  in  the  resampled  correlation  patch. 

After  the  correlation  coefficients  for  all  the  resampled  cor¬ 
relation  patches  have  been  calculated,  the  patch  with  the  largest 
correlation  coefficient  is  located.  Then,  a  parabolic  correlation 
function  is  fitted  to  the  largest  correlation  coefficient  and  the 
values  of  R  for  the  two  patches  on  either  side  of  the  patch  with  the 
largest  correlation  coefficient.  The  maxima  of  the  correlation 
function,  the  point  where  the  slope  of  the  function  is  zero,  gives 

the  location  of  the  match  point  to  the  nearest  pixel.  The  scheme 
described  above  is  shown  in  Figure  13. 
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When  the  patch  with  the  largest  correlation  coefficient  is  at 
either  end  of  the  correlation  search  area,  a  parabolic  correlation 
function  cannot  be  calculated.  The  center  of  the  end  patch  is 
assumed  to  be  the  match  point.  The  point  is,  however,  regarded  as 
suspect  and  is  a  candidate  for  further  processing. 

4.60  Block  Wandering  Tolerance 

In  hard  to  correlate  areas,  the  match  points  of  some  blocks  tend 
to  wander  with  respect  to  blocks  in  neighboring  block  paths.  To 
prevent  such  artificial  bulges  in  the  surface  at  these  match  points, 
the  U  coordinate  of  the  match  point  of  each  block  in  a  column  is 
compared  to  the  average  position,  Uav»  °f  the  match  points  of  the  two 
adjacent  blocks.  If  the  difference  |U-Uav|  exceeds  a  prescribed 
tolerance,  Ut ,  the  block  is  said  to  wander.  The  match  point  for  the 
wandering  block  is  moved  towards  the  neighbor  average  Uav  by  a 
weighted  amount.  The  corrected  match  point  is  given  by 

UC  *  U  +  W  (Uav  -  U)  (8) 

where  Uc  is  the  coordinate  of  the  corrected  match  point  and  W  the 
weight  for  wandering  block  correction.  The  correcting  scheme  is 
shown  in  Figure  17. 

The  value  of  the  local  rate  of  change  function  is  updated  to 
reflect  the  wandering  block  correction. 

4.70  Structure  of  the  Image  Matching  Software 

The  matching  algorithm,  composed  of  elements  described  in  Sect¬ 
ions  4.20  through  4.60,  is  outlined  in  the  flow  chart  in  Figure  18. 
The  figure  shows  how  the  elements  are  integrated  into  a  software 
package  for  automated  stereo-photogr ammetr y  using  SEM  images.  Each 
of  the  major  operations  have  been  divided  into  simple  functional 
modules  to  facilitate  changes  in  the  computer  code  in  accordance  with 
technological  advances  in  the  subject  areas  of  the  modules. 
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The  image  matching  program  is  set  up  to  match  one  block  path  at 
a  time.  A  complete  surface  is  mapped  by  matching  a  series  of 
adjacent  strips.  The  height  data  for  each  strip  is  stored  in  a  file 
on  disc.  Instructions  for  using  the  software  are  given  in  Section 

4.80. 

The  matching  program  has  a  few  control  variables  to  help  guide 
the  program  through  areas  that  are  hard  to  match.  The  two  parameters 
are  the  maximum  and  minimum  values  for  the  rate  of  change  function. 
If  the  local  rate  of  change  function  at  any  block  is  greater  than  the 
maximum  limit,  it  is  set  equal  to  the  maximum;  if  the  rate  of  change 
function  is  less  than  the  minimum  limit,  it  is  set  equal  to  the 
minimum.  Flags  are  set  to  indicate  that  the  value  of  the  rate  of 
change  function  was  not  within  the  specified  limits.  The  limiting 
values  for  the  rate  of  change  function  should  conform  with  the  range 
of  values  of  surface  slopes  expected  for  the  surface  being  mapped 
(see  Section  4.30) . 

The  matching  program  needs  the  screen  coordinates  of  one  pair  of 
matched  points  to  start  matching  the  block  path.  The  matched  point 
in  the  independent  image  is  set  as  the  center  of  the  first  block  in 
the  block  path  and  its  conjugate  point  in  image  B  marks  the  starting 
point  for  the  correlation  search  area  in  image  B. 

After  a  block  path  has  been  mapped,  the  matched  points  are 
checked  to  see  if  the  correlation  coefficients  are  greater  than  a 
minimum  acceptable  value  input  by  the  user;  a  point  with  the 
correlation  coefficient  less  than  the  minimum  limit  is  considered 
unacceptable.  If  the  number  of  unacceptable  points  exceeds  a 
specified  value,  the  block  path  is  matched  again  to  refine  the 
results.  During  the  refining  iterations,  the  coordinates  of 
matched  points  obtained  in  the  previous  iteration  are  used  as  the 
predicted  locations  of  the  match  points  in  image  B;  the  rate  of 
change  function  is  not  used  for  prediction  (see  Section  4.30)  .  The 
refining  iterations  are  continued  until  either  the  number  of  un- 
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acceptable  points  is  less  than  the  specified  value  or  the  number  of 
iterations  equal  a  user  specified  limit. 

The  coordinates  of  matched  points  are  used  to  calculate  the 
height  of  the  centers  of  all  the  blocks  in  the  block  path  relative 
to  the  height  of  the  first  block.  For  a  parallel  projection  geometry 
in  the  SEM,  the  height  of  a  block  i  relative  to  the  first  block  in 
the  path  is  given  by 

z.  =  d(  xLcosaR  -  xrcosaL  )  (9) 

sin  (aR  -  a  L) 


I 


where  xL  is  the  distance  between  block  i  and  block  1  in  image  A,  xR 
the  distance  between  block  i  and  block  1  in  image  B,  d  the  distance 
between  pixels  (see  Eq .  1)  in  image  A,  the  angle  of  tilt  for  image 
A,  and  ^  the  angle  of  tilt  for  image  B.  Lengths  x^  and  xR  are  measured 
in  screen  coordinate  units. 

4.80  Instructions  for  Use 

The  program  input  consists  of  the  variables  for  the  size  of 
:  i  '.'-'.s  and  correlation  patches,  control  parameters  to  guide  the 
matching  algorithm,  coordinates  of  the  matched  points  for  the  block 
path  being  matched,  and  the  name  of  the  files  in  which  the  digitized 
SEM  images  are  stored.  All  the  other  information,  for  example  the 
magnification,  is  read  from  header  blocks  in  the  image  files. 

The  input  parameters  for  blocks  and  patches  are  the  width  of  a 
block  along  the  X  axis  (NX),  the  number  of  scan  lines  in  the  block 
path  (NY)  ,  number  of  blocks  in  the  block  path  (NCOLUM)  ,  the  distance 
(in  number  of  pixels)  between  the  centers  of  adjacent  blocks  in  the 
path  (NDIST) ,  and  the  number  of  correlation  patches  in  the  search 
area  in  image  B  (INDEX).  The  control  parameters  are  the  minimum 
acceptable  value  of  the  correlation  coefficient  (CORMIN) ,  the  number 
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of  acceptable  points  allowed  (NAXEPT) ,  the  maximum  number  of  iter¬ 
ations  allowed  (NLOOPS),  the  maximum  value  of  the  rate  of  change 
function  (RATMAX)  ,  and  the  minimum  value  of  the  rate  of  change 
function  (RATMIN) .  The  remaining  input  data  are  the  names  of  the 
data  files  in  which  the  stereo-pair  of  images  are  stored  and  the 
screen  coordinates  of  the  pair  of  matched  points  provided  to  start 
the  algorithm.  The  height  data  for  the  block  path  are  stored  in  a 
file  with  the  same  name  as  the  independent  image  but  a  file  extension 
HIT.  The  input  variables  are  listed  in  Table  3  along  with  the  range 
of  acceptable  values  for  the  variables. 

5 . 0  VERIFICATION  OF  INTEGRATED  SYSTEM  (TASK  4) 

A  photograph  of  the  computer  SEM  system  is  shown  in  Figure  19. 
The  procedure  for  mapping  a  surface  using  the  surface  measurement 
system  is  as  follows: 

1)  Tilt  the  specimen  at  31.469° 

2)  Digitize  the  area  of  interest 

3)  Tilt  the  specimen  at  40.485° 

4)  Relocate  the  area  of  interest  and  digitize  the  second 
image . 

5)  Match  the  stereo-pair  of  images 

The  stop-block  arrangement  to  obtain  tilt  angles  of  31.469° 
and  40.485°  was  developed  in  Phase  I  of  the  program.  The  arrange¬ 
ment,  shown  in  Figure  20,  was  necessary  because  the  tilt  angles  must 
be  known  accurately  for  calculating  the  heights  of  points  based  on 
the  differences  in  parallax.  The  tilt  mechanism  in  the  SEM  was  not 
accurate  enough.  The  estimated  reproducibility  of  the  stop-block 
arrangement  was  +  0.1°  from  the  average  tilt  angle.  The  choice  of 
the  two  angles  was  dictated  by  the  physical  constraints  of  the  SEM 
chamber  and  stage.  The  difference  of  9  degrees  between  the  tilt 
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angles  was  selected  because  it  approximates  the  angle  subtended  by 
human  eyes  when  viewing  objects  at  distance  of  25  cm. 

5.10  Verification  of  Digitized  Images 

The  quality  o!  the  digitized  images  was  verified  hv  display¬ 
ing  a  digit ized  image  on  the  SUM  view  CRT  and  phot og raph i ng  the  image 

Figure  21  shows  a  SEM  micrograph  of  a  hardness  impression  and  the 
digitized  image  for  the  same  surface.  Both  images  were  scanned 
under  computer  control  to  obtain  images  with  a  resolution  of  512  x 
256  pixels.  Figure  22  shows  512  x  512  pixel  images  of  a  corrosion 
pit.  From  Figures  21  and  22  it  is  evident  that  the  digitized  images 
are  a  faithful  reproduction  of  the  usual  SEM  micrographs.  The 
contrast  and  brightness  levels  differ  because  while  the  SEM  micro¬ 
graphs  are  processed  for  the  best  visual  impression,  the  digitized 
gray  levels  are  optimized  for  use  with  the  image  matching  software. 

A  512  x  256  pixel  digitized  image  of  a  low  contrast  sample  is  shown 
in  Figure  23  along  with  a  SEM  micrograph  with  a  resolution  of  1024 
scan  lines.  Again,  the  digitized  image  reproduced  most  of  the 
finer  detail  on  the  surface  despite  the  low  contrast.  For  most 
surfaces,  a  resolution  of  512  x  512  pixels  is  recommended.  A  higher 
resolution  should  be  used  only  for  busy  surfaces. 

5.20  Verification  of  the  Surface  Measurement  System 

The  surface  measurement  system  was  verified  by  matching  a 
stereo-pair  of  digitized  micrographs  of  the  el ectropol ished  alumi¬ 
num  specimen  used  in  Phase  I  for  the  proof-of-concept  tests.  A 
schematic  of  the  specimen  is  shown  in  Figure  24  and  the  stereo-pair 
of  digitized  images  are  shown  in  Figure  25. 

The  image  matching  software  was  used  to  obtain  heights  along 
grid  line  6.  The  results  for  two  different  sets  of  limits  for  the 
rate  of  change  function  are  shown  in  Figure  26  along  with  an  actual 
stylus  trace  and  the  results  obtained  by  matching  points  by  eye. 
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The  height  measurements  using  a  tight  set  of  restrains  on  the  rate 
of  change  function  are  in  good  qualitative  agreement  with  the  stylus 
trace  and  the  results  from  eye  measurements.  The  stylus  measure¬ 
ments  were  made  with  a  stylus  having  a  25  Mm  tip  radius  and  will, 
therefore,  underestimate  the  depth  of  the  groove  at  grid  line  2 
because  the  stylus  will  not  enter  the  narrow  regions  of  the  groove. 
The  trace  from  the  matching  software  has  more  noise  than  the  stylus 
trace.  The  noise  may  be  due  to: 

1)  An  artifact  of  the  matching  algorithm 

2)  Features  not  present  along  the  line  traced  by  the  stylus 
but  present  along  grid  line  6. 

3)  Features  not  detected  by  the  stylus. 

More  experimental  work  needs  to  be  done  to  identify  the  sources 
of  noise  in  the  image  matching  software.  The  matching  algorithm  is 
purely  statistical  and  fails  in  areas  with  very  little  contrast  or 
areas  with  too  much  gray  level  information.  The  experimental  work 
should  focus  on  identifying  variables  that  will  anticipate  the 
possibility  of  failure  of  the  matching  algorithm.  Control  strate¬ 
gies  can  then  be  devised  to  guide  the  matching  algorithm  through 
areas  that  are  hard  to  match. 


SKF  TECHNOLOGY  SERVICES 

Skf  INHUSTW  S  ink 


AT 8  3 DO 0  4 


6 . 0  SUMMARY  AND  CONCLUSIONS 

The  integration  of  software  and  hardware  for  digitizing  SEM 
images  has  been  completed.  The  system  reliability  has  been 

verified  by  experiments  designed  to  check  the  quality  and  repeat¬ 
ability  of  digitized  image  data.  Procedures  have  been  developed 
to  obtain  digitized  SEM  images  suitable  for  matching  a  stereo-pair 
of  images.  The  image  matching  software  is  designed  to  process  a 
single  strip  of  an  SEM  image.  The  complete  surface  can  be  mapped 
by  processing  a  series  of  adjacent  strips.  The  integrated  system  has 
been  used  successfully  to  obtain  quantitative  height  data  from  a 
stereo-pair  of  SEM  images.  The  image  digitization  system  can  be 
regarded  as  a  reliable,  mature  system.  The  image  matching  software 
is,  however,  still  in  the  development  stage.  The  major  drawback  is 
the  reliance  on  purely  statistical  measures  of  correlation  to  locate 
the  match  points.  The  matching  software  needs  some  intelligence  in 
t  h<_  form  of  heuristic  control  algorithms  to  guide  the  matching 
p: ocess  in  areas  that  are  hard  to  match. 

The  SEM  based  system  has  the  following  advantages  over  a  typical 
.:•>  y  l  us  system: 

1)  The  SEM  system  is  non-contacting  and  can,  therefore,  be  used 
.v  i  :•  h  soft  surfaces  (tor  example,  coated  surfaces)  which  may 
:><■  damaged  by  a  stylus. 

2)  The  SEM  system  has  a  horizontal  resolution  at  least  an  order 
of  magnitude  greater  than  the  best  stylus  instruments.  The 
system  can,  therefore,  be  used  to  map  surface  features  which 
the  stylus  instruments  are  not  capable  of  observing.  Ex¬ 
amples  of  such  surface  features  are  spalls  and  cracks  on 
bearing  surfaces,  details  of  fracture  surfaces,  etc.  The 
shape  of  such  surfaces  provides  important  information  to¬ 
wards  identifying  the  cause  of  spalls  and  fractures. 

3)  Output  from  any  of  the  SEM  sensors  (x-ray,  backscatter 
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electrons,  secondard  electrons,  etc.)  can  be  digitized  using 
this  system.  The  system  can,  therefore,  be  used  to  provide  a 
map  of  surface  materials  along  with  the  surface  topography. 

The  stereo-pairs  surface  topography  system  needs  further  de¬ 
velopment  to  evolve  into  a  handy,  reliable  surface  measurement 
system.  The  work  should  focus  on  improving  the  reliability  and  the 
stability  of  the  image  matching  software. 
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7 . 0  SCOPE  OF  FURTHER  DEVELOPMENT  EFFORT 

Image  Processing 

The  image  matching  software  uses  gray  level  differences  to 
locate  match  points.  A  number  of  tribological  surfaces  yield 

images  with  poor  contrast.  Image  enhancement  software  to  improve 
contrast  will  improve  the  performance  of  the  matching  software.  At 
the  other  extreme,  too  much  gray  level  information  in  the  image  can 
defeat  the  matching  algorithm.  Fourier  analysis  software  could  be 
used  to  filter  out  the  very  high  frequency  variations  in  the  gray 
level.  The  filter  will  be  used  in  the  same  way  as  the  stylus 
instruments  use  filters  to  separate  waviness  and  roughness. 

Image  Matching 

The  current  matching  software  should  be  used  to  match  a  wide 
range  of  surfaces  to  accumulate  empirical  data  on  control  strategies 
required  to  match  images.  The  data  can  then  be  used  to  develop 
control  algorithms  to  guide  the  matching  program  in  areas  that  are 
hard  to  match.  The  lack  of  adequate  general  purpose  control 
algorithms  is  the  primary  reason  for  the  problems  encountered  in 
matching  images. 

Suv  face  Analysis 

Most  of  the  surface  analysis  software  currently  available  is 
for  two-dimensional  surface  profiles.  Surface  analysis  software 
should  be  developed  to  exploit  the  three-dimensional  surface  height 
measurement  capability  of  the  stereo-pairs  surface  topography  sys¬ 
tem.  The  software  should  include  mul t i -d 1  mens ional  spectral 

analysis,  multi-dimensional  digital  filters,  identification  of  sur¬ 
face  features  of  interest  (summits,  valleys,  etc.),  surface  height 
distributions,  contour  maps,  etc. 
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A  partial  list  of  references  on  digital  photogr ammetry  is  pro¬ 
vided  below.  A  comprehensive  list  of  references  on  SEM  stereology 
and  photogrammetry  is  included  in  the  final  report  for  Phase  I  of  the 
program  (Gassel  et  al  1983). 
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TABLE  1  RANGE  OF  ACCEPTABLE  VALUES 
FOR  THE  INPUT  VARIABLES 


RANGE  OF  VALUES 


NXSCAN 

NYSCAN 

NROW 

NPIXEL 

MAGNFY 

CVOLTS 

NAV 

TILT 


1  to  1024 
1  to  1024 

1  to  1024  (<  NYSCAN) 

256,  512,  768,  or  1024  (<  NXSCAN) 
500X  to  32 , 000X 
30,000  Volts  Maximum 
1  to  15 

0  to  60  Degree 


TABLE  2  DEFAULT  VALUES  FOR  PARAMETERS 
NOT  INPUT  BY  THE  USER 


PARAMETER  DEFAULT  VALUE 


IDWELL 

200  ys 

NXLEFT 

(NXSCAN 

-  NPIXEL) /2 

NYTOP 

(NYSCAN 

-  NROW) /2 

W 

88.9  mm 

I. 

1. 

I. 


[ 
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TABLE  3  INPUT  VARIABLES  FOR  IMAGE  MATCHING  SOFTWARE 


VARIABLE 


RANGE  OF  ACCEPTABLE  VALUES 


NX 

NY 

NCOLUM 

NDIST 

INDEX 

CORMIN 

NAXEPT 

RATMAX 

R ATM IN 

NLOOFS 


5  to  15 
5  to  15 
<  128 
>  1 
5  to  21 
0  to  1.0 
>  1 
>  0 
>  0 
>  1 
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FIGURE  2  SCREEN  COORDINATE  SYSTEM 
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FIGURE  3  FLOW  CHART  FOR  THE  MAIN  CONTROL  PROGRAM 
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FI.OW  CHART  TOR  DIGITIZING 
SRM  IMAGES 
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DATA  COLLECTION  SOFTWARE 

MAKE  SURE  THAT  THE  'CAPS  LOCK'  KEY  IS  NOT  DEPRESSED 
TO  DISPLAY  THE  CURSOR  BOX: 

1)  SET  THE  MANUAL  VIDEO  AMPLIFIER  TO  'SUM' 

2)  SET  THE  CHANNEL  SELECT  FOR  1 XRAY2 ' 

3)  REVERSE  POLARITY  OF  SIGNAL 

4)  SET  PHOTO  CRT  FOR  'CHANNEL  2' 

PRESS  RETURN  TO  CONTINUE 


FIGURE  5  START-UP  INSTRUCTIONS 
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DATA  COLLECTION  SOFTWARE 

NO. 

FUNCTION 

1 

SELECT  AREA  TO  BE  DIGITIZED 

2 

SET  INTEGRATION  TIME 

3 

OBTAIN  COORDINATES  OF  POINTS 

OF  INTEREST 

4 

DIGITIZE  AND  STORE  IMAGE 

5 

STOP 

ENTER 

FUNCTION  NO.: 

FIGURE  6  FUNCTION  SELECTION  MENU 
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POSITION  THE  CURSOR  BOX 


TO  MOVE  THE  BOX: 

1)  PRESS  THE  PF3  KEY 

2)  USE  THE  ARROW  FUNCTION  KEYS  TO 
MOVE  THE  BOX 


TO  CHANGE  THE  SIZE  OF  THE  BOX: 

1)  PRESS  THE  PF2  KEY 

2)  USE  THE  ARROW  FUNCTION  KEYS  TO 
MOVE  THE  LOWER  EDGE  OF  THE 
CURSOR  BOX  UP  OR  DOWN 


PRESS  PF4  KEY  WHEN  FINISHED 


FIGURE  7  CURSOR  BOX  MENU 
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FIGURE  8  INTEGRATION  TIME  MENU 
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SELECT  POINTS  OF  INTEREST 

USE  THE  ARROW  FUNCTION  KEYS  TO 
MOVE  THE  CROSS-HAIR  CURSOR  TO 
THE  POINT  OF  INTEREST 

PRESS  C  KEY  TO  GET  COORDINATES 

PRESS  PF4  KEY  WHEN  FINISHED 


FIGURE  9  POINT  SELECTION  MENU 
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CORRELATION 

—  SEARCH  AREA  - 

SHAPED  CORRELATION  PATCHES 

/  ,  \  \ 


POINT  (  X ,  Y)  TO 
BE  MATCHED 


PREDICTED 
MATCH  POINT 


RESULT  OF  MATCHING  IS  A  5 -TUPLE! 


(x,  y,  utv,r) 


COORDINATES 
OF  POINT  ON 
IMAGE  A 


7IGURE  12 


COORDINATES 
OF  POINT  ON 
IMAGE  3 


RELIABILITY 
FACTOR  OF 
THE  MATCH 


BLOCK  (I,J)  AND  TT-S  CORRELATION 
SEARCH  AREA  ON  IMAGE  B 
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MAXIMUM  COMPUTED 
CORRELATION  VALUE 


COEFFICIENT 


FIGURE  13  DETERMINATION  OF  THE  MATCH  POINT 


AT83D004 


FIGURE  14  EFFECT  OF  SURFACE  TOPOGRAPHY  ON 
THE  RATE  OF  CHANGE  FUNCTION 


AT  8  3  D  0  0  4 


'K 


f I CURE  15  GRID  FOR  CALCULATING  THE  RATE  OF  CHANGE  FUNCTION 
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BLOCK  (I,J)  ON 
IMAGE  B 


SHAPED  PATCH  ON 
IMAGE  B 


(a)  PATCH  SHAPING 


PIXEL  RESAMPLED 

1  POINT 

-e - g - 

k— Dx  -  1-DX 


1 


PIXEL 

2 

— e- 


(b)  RESAMPLING 


FIGURE  16  PATCH  SHAPING  AND  RESAMPLING 
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WANOERING 
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neighbor 

AVERAGE 


•  BLOCK  IS  CORRECTED  TOWARD  NEIGHBOR 
AVERAGE  BY  A  WEIGHTED  AMOUNT 

•  WEIGHT  DETERMINED  BY  TUNING 


FIGURE  17  WANDERING  BLOCK  TOLERANCE 
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FIGURE  18  FLOW  CHART  FOR  THE  IMAGE  MATCHING 
SOFTWARE 
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APPENDIX  A 


VALIDATION  OF  SOFTWARE  USED  TO 
DIGITIZE  SEM  IMAGES 
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1.0  INTRODUCTION 

Early  attempts  to  match  a  stereo-pair  of  digitized  SEM 
images  yielded  very  poor  correlation  between  the  two  images . 

An  examination  of  the  digitized  image  data  revealed  substantial 
differences  in  the  digitized  values  for  the  stereo-pair  of  im¬ 
ages.  Subsequently,  the  same  area  on  a  specimen  was  digitized 
twice  to  check  the  repeatability  of  the  data  collection  system. 
The  results  showed  very  poor  repeatability.  A  series  of  tests 
were  designed  to  identify  the  causes  of  the  errors  and  to  de¬ 
velop  a  procedure  that  would  result  in  a  stereo-pair  of  digitized 
SEM  images  optimized  for  the  image  matching  software. 


Appendix  C  describes  the  tests  conducted  to  verify 
the  repeatability  of  digitized  image  data  collected  using  DATCOL 
software  package  described  in  Section  3  of  the  report.  The 
tests  showed  that  the  lack  of  repeatability  was  primarily  due  to 
dynamic  instabilities  in  the  X-Y  scan  coils  of  the  SEM.  The  prob¬ 
lem  was  corrected  by  waiting  long  enough  for  the  beam  to  stabi¬ 
lize  at  each  point  before  the  signal  is  digitized.  The  tests 
led  to  a  definition  of  a  procedure  to  obtain  digitized  SEM  images 
suitable  for  matching  a  stereo-pair  of  digitized  images. 


2 . 0  TEST  PROCEDURE 


The  same  area  on  a  specimen  was  digitized  five  times.  Each 
of  the  five  digitized  images  was  an  array  of  512  X  20  pixels. 

The  five  images  were  used  to  calculate  the  standard  deviation  a 
and  the  average  gray  level  s  at  each  of  the  10,240  pixels  in  an 
image.  The  standard  deviation  was  used  as  a  measure  of  the  noise 
at  each  point  and  used  to  calculate  the  noise-to-signal  ratio, 

N/S ,  given  by 


N/S  =  a/s  (1) 

The  results  are  presented  as  plots  of  N/S  versus  s  for  each  of 
the  10,240  points  in  the  image. 
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3 . 0  TEST  RESULTS 


3.1  Base-line  Results 


The  base-line  results  were  obtained  using  the  old  procedure 
for  integration  times  of  20  ps  (Figures  1  and  2)  and  100  ps  (Fig¬ 
ures  3  and  4) .  Figures  1  and  3  showed  that  the  noise  in  the  data 
was  essentially  independent  of  the  magnitude  of  the  signal.  The 
dwell  time  determined  the  range  of  gray  levels  in  the  data  but 
did  not  effect  the  distribution  of  noise.  For  gray  levels  be¬ 
tween  -200  and  +200,  the  N/S  ratio  was  greater  than  50  percent 
(Figures  2  and  4) . 

The  noise  in  the  data  was  attributed  primarily  to  two 
causes : 

1.  Dynamic  instability  in  the  X-Y  scan  coils  of  the  SEM 
resulting  in  small  deviations  in  the  position  of  the 
beam.  For  images  in  the  test  (magnification  =»  1000X)  , 
the  probe  diameter  is  less  than  the  distance  between 
pixels.  Therefore,  a  deviation  in  beam  position  will 
result  in  errors  because  a  different  part  of  the  speci¬ 
men  is  sampled. 

2.  The  SEM  signal  is  generated  by  the  interaction  of  the 
electrons  in  the  beam  and  the  specimen  material.  The 
beam-specimen  interaction  is  a  random  process  and  there¬ 
fore,  some  noise  is  inherent  in  the  signal. 

3 . 2  Remedies  Examined 


The  following  procedures  were  tested  to  reduce  the  N/S 
ratio  and  to  improve  the  repeatability  of  the  data : 

1.  Digitize  images  such  that  the  gray  levels  are  greater 
than  200. 

2 .  Allow  more  time  for  the  beam  to  settle  at  each  point 
before  integrating  and  digitizing  the  signal. 

3.  Scan  each  line  in  the  image  repeatedly  to  calculate  an 
average  gray  level  at  each  point  in  the  line.  The  num¬ 
ber  of  samples  at  each  pixel  in  the  image  will  be  selec¬ 
ted  to  yield  the  minimum  N/S  ratio.  The  average  will 
help  to  attenuate  the  random  noise  in  the  signal. 
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4.  Increase  the  signal-to-noise  ratio  of  the  SEM  signal 
by  increasing  the  beam  current.  This  is  achieved  by 
decreasing  the  current  in  the  condenser  coils  from  the 
usual  2.0  amps  to  1.7  amps.  The  result  is  an  enhanced 
SEM  signal  and  a  larger  probe  diameter.  The  larger 
probe  diameter  provides  some  overlap  between  pixels  and 
helps  reduce  the  magnitude  of  change  in  gray  levels 
caused  by  small  deviations  in  the  position  of  the  beam 
due  to  X-Y  scan  coil  instabilities.  The  probe  diameter 
should,  however,  be  less  than  1.5  times  the  distance  be¬ 
tween  pixels  to  avoid  introducing  new  errors. 

3.3  Results  of  Modifications 


Figure  5  shows  the  effect  of  increasing  the  beam  setting 
time  between  scan  lines .  The  N/S  ratio  is  reduced  to  half  of 
the  values  in  Figure  4.  The  N/S  ratio  is  improved  significantly 
when  each  gray  level  is  an  average  of  five  samples  (Figure  6) 
but  begins  to  degrade  when  ten  samples  are  averaged  (Figure  7) . 
The  existence  of  an  optimum  number  of  samples  per  pixel  was  con¬ 
firmed  by  repeating  the  tests  using  other  samples.  The  results 
suggested  that  some  systemic  causes  other  than  random  noise  were 
contributing  to  the  variation  in  gray  levels.  Further  tests 
showed  that  the  specimen  needs  time  to  recover  from  the  effects 
of  the  electron  beam.  If  the  same  spot  on  the  specimen  is  sam¬ 
pled  repeatedly  without  allowing  sufficient  time  between  samples, 
the  cumulative  effects  of  the  beam-specimen  interaction  result  in 
a  decrease  in  gray  level.  The  level  begins  to  stabilize  after 
40  to  50  samples.  Therefore,  multiple  samples  for  calculating 
average  gray  levels  should  be  taken  such  that  there  is  sufficient 
time  between  samples  for  the  effects  of  beam-specimen  interaction 
to  dissipate. 

Figures  8,  9,  and  10  show  the  effect  of  increasing  the  beam 
signal  by  reducing  the  current  in  the  condenser  coils  and  further 
changes  in  scan  procedure  to  reduce  dynamic  instabilities  in  X-Y 
scan  coils.  There  is  a  significant  reduction  in  the  N/S  ratio. 
The  degradation  in  data  for  10  samples  per  pixels  is  present  be¬ 
cause  the  modifications  discussed  above  had  not  yet  been 
incorporated . 

4.0  CONCLUSIONS 


The  reduction  in  N/S  ratio  due  to  improved  X-Y  scan  coil 
stability  are  sufficient  to  avoid  having  to  adjust  the  condenser 


58 


SKF  TECHNOLOGY  SEPVCES 

SKF 'NDjSTP'ES  nc 


current  to  obtain  good  digitized  data.  For  most  surfaces,  the 
following  procedure  will  yield  acceptable  images: 


1 .  Ensure  that  the  minimum  gray  level  is  above  +200.  The 
data  collection  software  was  modified  to  set  the  dwell 
time  automatically  to  ensure  that  the  data  are  spread 
over  the  full  positive  range  of  gray  levels  (max.  2047) . 
If  the  SEM  contrast/brightness  controls  are  adjusted  to 
keep  the  waveform  above  the  zero  level,  the  digitized 
data  are  in  the  200  -  2047  range. 

2.  The  default  is  only  one  sample  per  pixel .  For  most 
specimens,  averaging  will  not  be  required  if  the  gray 
levels  are  between  200  -  2047.  When  it  is  not  possible 
to  satisfy  the  above  requirement,  five  samples  per  pixel 
should  be  used. 

3 .  Condenser  coil  current  adjustment  should  be  used  only 
as  a  last  resort. 

Images  collected  using  the  above  criteria  were  consistently 
of  good  quality.  Digitized  data  obtained  from  the  same  surface 
varied  by  less  5  percent.  Preliminary  attempts  to  match  stereo¬ 
pairs  collected  using  the  new  procedure  yielded  dramatic  improve¬ 
ments  in  correlation  between  the  two  images. 
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